Causal Effects and Experiments

Causes and effects

Our goal is to better understand what factors lead to certain outcomes of interest.

• Does increasing the number of voting booths close to a potential voter make that person more likely to vote?

• Do peace treaties signed with factionalized rebel groups more often lead to a return to conflict than those signed with a single, cohesive group?

• Does trade between two countries make war between them less likely?

Causes and effects

These are causal statements:

• More local voting booths \(\rightarrow\) More likely to vote

• More factionalization \(\rightarrow\) More likely to restart conflict

• Trade \(\rightarrow\) Less likely to go to war

How to identify causes and their effects

Proving that changes to one factor cause changes to another is very tricky!

• Need to account for all the other factors

The efficacy of international election monitors

Do international monitors cause less election-day fraud in democratic elections?

• Number of elections monitored by international observers exploded throughout the 2000s

• But do monitors increase the chances the election will be free and fair?

• In other words, are they effective?

The efficacy of international election monitors

Dr Susan Hyde set out to answer this very question. From her article (page 39):

If the presence of international observers causes a reduction in election-day fraud, the effect of observers should be visible at the subnational level by comparing polling stations that were visited by observers with those that were not visited. More specifically, if international monitoring reduces electionday fraud directly, all else held equal, the cheating parties should gain less of their ill-gotten vote share in polling stations that were visited by international monitors.

Causal relationships

Refers to the directional connection between a change in one variable and a corresponding change in another.

• The direction matters!

• Treatment variable: the variable causing changes to another variable.

• Outcome variable: the variable changing as a result of changes to another variable (the treatment).

Treatment and outcome variables

We want to test whether the presence of international monitors (treatment) leads to less election day fraud (outcome).

International election monitors present at polling stations \(\rightarrow\) Less election-day fraud at that station

Treatment variable

At any given polling station in any given election, monitors may be: 1) present, or 2) not present.

Two different conditions:

• Treatment: condition with treatment (monitors are present)

• Control: condition without treatment (monitors are not present)

Outcome variable

At any given polling station in any given election, fraud may: 1) occur, or 2) not occur.

Observing our outcome

Sometimes it can be hard to observe our outcome of interest.

• How might we see all election-day fraud?

• How might we measure it?

Observing our outcome

Hyde’s answer: vote-share!

International election monitors present at polling stations \(\rightarrow\) Less election-day fraud at that station \(\rightarrow\) Lower vote-share for the cheating party

Individual causal effects

We want to know whether the treatment causes a change in our outcome of interest.

What might this look like in the real world?

• Imagine we are looking at a specific election

• Five polling stations

Parallel worlds

Hypothetical election

Individual effects

The only difference between these two conditions is the presence of international monitors!

• The difference between vote shares under these conditions is caused by the monitors.

Individual effects

No parallel worlds

Sadly for us, we cannot create parallel worlds…

What now?

Average causal effects

We need to move away from looking at individuals and start to look for patterns in our group.

Back to our parallel worlds

Difference of averages across all individuals

What was the average vote share received in each world?

Back to reality

Difference-of-means

What was the average vote share received in each group?

How on Earth does this work so well?

Randomization!

• Monitors were assigned to polling stations randomly (for example, with the flip of a coin)

• This created two groups of stations that were roughly identical on average to one another prior to treatment

• This mimics what happens when we split our world into two (creating two literally identical groups)

Our goal

We want two groups that are completely identical to one another prior to treatment.

• This allows us to compare their outcomes after treatment and claim that the treatment caused differences in those outcomes (if any differences exist)

Our goal

Next best thing

Instead, we should try to make two groups that are as similar as possible to each other prior to treatment.

The magic of randomization

Perfectly random assignment does this very well!

Don’t take my word for it

Imagine we have a group of 1,000 individuals. We know the following about them:

• Height

• Weight

• Eye colour

Our group

Random assignment

I’m now going to flip (an imaginary, computer-generated) coin for each of these 1,000 individuals to assigned them to group A or B: